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SIMULATION MODELING 
AS A POLICY TOOL 


Bernardo Alves Furtado 


This chapter describes, justifies, presents the pros and cons of and illustrates the use of simula- 
tion modeling as a handy, cost-effective and agile tool for policymakers. Simulation modeling 
is flexible enough to accommodate different levels of detail, precision and time frameworks. It 
also serves the purpose of a concrete communication platform that facilitates scenario analysis, 
what-if alternatives and forward looking. We specifically define agent-based modeling within 
the larger simulation domain, provide a brief overview of other computation modeling meth- 
odologies and discuss the concepts of multiple models, verification, validation and calibration. 
The conceptual framework section closes with a discussion of advantages and disadvantages of 
using simulation modeling for policy at various stages of implementation. Finally, we present 
a panorama of actual applications of simulation modeling in policy, with an emphasis on eco- 
nomic analysis. 


Policy as a Complex System 


Policymakers face the arduous task of balancing interests, pressure, bureaucracy and evidence 
to foster a construed consensus of social betterment. In fact, more often than otherwise, there 
are disagreements about what the priorities are, which goals to seek, what methods are available 
and, if correctly and fully implemented, which endings will result (Gaus 2021). This descrip- 
tion shares a number of characteristics with a complex system definition (Furtado and Sakowski 
2014). 

Policies and their effects thereafter are obtained from the interaction of heterogeneous agents: 
citizens, sub-groups, bureaucrats, interested parties, lobbyists, activists, leaders and their contex- 
tual environment. These interactions among different actors produce emerging results embed- 
ded with non-linearities that diffuse across scales in time and space with distinct levels of rigidity 
and friction. Moreover, those who benefit and those who suffer negative externalities react. 
Together, such systems evolve, adapt, change and learn. 

This understanding that policymaking may involves complex systems implies that they are 
hard to predict (Mitchell 2011; Polhill et al. 2021). Furthermore, full understanding of a single 
mechanism does not guarantee the comprehension of the whole: “the whole becomes not only 
more but very different from the sum of its parts” (Anderson 1972, 395). 
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Considering that policymaking operates and is intrinsically entangled with a social complex 
system suggest that a conceptual view of the system is needed and that appropriate methodolo- 
gies should be applied when handling policy interventions. Bettencourt (2015) illustrates how 
some policy problems may accept linear, additive solutions whereas others, classified as wicked 
or complex, may not. Traffic flow through an intersection is a problem that specific equations 
and variables that account for flow and speed might lead toward an optimal and unique solution. 
Urban mobility and social violence, however, are phenomena that involve multiple actors and 
themes for which there may be not a single solution. Instead, a pathway of possible alternatives, 
at times contradictory, might or might not be achieved, depending, for example, on how parties 
engage, react, contribute or participate. 

This scenario that policymakers face makes simulation in general and agent-based modeling 
in particular suitable tools. Even more so when considering the growing availability of data, 
computational power and the need for interdisciplinary teams (Polhill et al. 2021), because 
simulation provides teams with a platform for communication, among other things: a common 
language that functions as a repository for concepts, understandings and behaviors that neces- 
sarily will have to function together to produce results. Moreover, in solving explicitly designed 
problems systemically, iteratively, over trial and errors, it helps foster communication among 
participants. Gilbert et al. (2018, 1) argue “the main benefit of designing and using a model is 
that it provides an understanding of the policy domain, rather than the numbers it generates”. 

Simulation of policy interventions provides two further advantages besides enabling a com- 
mon platform for communication. Simulation focuses on prognosis, analyzing future pathways 
and doing so via comparison with nonexistent alternatives, the so-called counterfactuals. Not 
being able to use experiments hinders social sciences when compared with natural sciences. A 
health policy analyst cannot ‘test’ a vaccination policy and evaluate ex post whether the decision 
was positive or not. Even before policy, pilots are tried out, or randomized-control trials are 
applied (Gilbert et al. 2018); ethical warrants are needed to insure no harm comes to citizens or 
to the environment from either the policy intervention or its omission. 

As simulations happen within computational environments, on top of artificial societies of 
their own (J. M. Epstein and Axtell 1996), they provide enough room for social scientists to 
experiment and test alternatives in a safe space. These experiments — these simulations — serve to 
evaluate possible scenarios, to explain broad consequences and to delineate the space of possibili- 
ties. As a result, simulations contribute by anticipating outcomes of policy actions. 

Epstein and Axtell (1996) proposed these artificial societies along with the generative con- 
cept for social sciences. Their motto was later set as ‘If you didn’t grow it, you didn’t explain it’ 
(Joshua M. Epstein 2007, 8). 

The context to which a simulation tool may apply is also relevant (Edmonds et al. 2019; 
Edmonds and Meyer 2017). Depending on the purpose of the model and the stakeholders 
involved, different validation expectations arise. Models that claim prediction, for instance, 
should provide results that match out-of-sample data: anticipating data that is unknown to the 
modelers and has not been used in the model itself. 

Edmonds et al. (2019) suggest seven purposes for simulation models, from the more specific 
to the more general: prediction, explanation, description, theoretical exploration, illustration, 
analogy and social learning. When it comes to the most abstract, social learning, the aim of 
the model is solely to function as a mediator, a platform that members of a group use to share 
information, concepts and meaning about a common problem. Therefore, no anticipation of 
accurate data is demanded for the model to achieve its set goal. 

In a broader context, simulations are models, abstractions of complex phenomena that mod- 
elers run to gain insights and systematic outputs. Models suffer criticism as they are based on 
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assumptions, theories and simplifications. However, in general, they are considered useful (Box 
and Draper 2007). Simulations do not demand exclusiveness on the policy phenomena analysis. 
Quite the contrary: there is evidence that multiple models are complimentary and may together 
encompass a more comprehensive understanding (Page 2012). 

Besides this introduction, the next section defines agent-based modeling and section 2 pres- 
ents some applications of ABM for policy. 


Agent-Based Modeling (ABM) as a Policy Tool 


Poledna et al. (2020) claim that Enrico Fermi made the first use of the agent-based model 
(ABM). In the 1930s, Fermi analyzed neuron transport by hand. Orcutt implemented an 
economic model in the late 1950s. However, the famous first model is considered to be 
the racial segregation analysis of neighborhoods made by Schelling (1969, 1972). The first 
attempt to make a systematic construct of concepts and inaugurate social modeling could be 
pinned to the seminal book by Epstein and Axtell (1996). In a similar pathway in economics, 
Tesfatsion and Judd (2006) coined the foundational agent-based computational economics 
(ACE). 

A general formulation of the concept of an agent-based model used to describe an artificial 
society (J. M. Epstein and Axtell 1996) is: 


Am = f(A,,E,) 
En = g(A,E,) 


which is a “discrete time dynamical system” (J. M. Epstein and Axtell 1996, 19) in which the 
states of agents and the environment in time f+1 depend on agents and the environment in t. 
How the transformations occur depend on the rules that are codified as f(.) and g(.). In practice, 
a simulation attempts to apply theoretical and tacit rules to data and “animate” (Galan et al. 
2009) the agents in time and space to obtain patterns that are empirically observed. 

When defined in contrast with models in which the “whole population . . . is collapsed into a 
single set of variables, [ABM can be defined as] computer simulations where individual elements 
of the social system are represented as separate elements in the simulation model” (Edmonds and 
Meyer 2017, 3). 

These definitions of ABM encompass its most useful features and delineate where their appli- 
cation is fruitful. Given the emphasis on the heterogeneity of agents, phenomena that include a 
number of different agents interacting in a decentralized way (policy arenas, for example) may 
be a suitable use. Moreover, if those agents are autonomous and active, in the sense that they 
learn, evolve, adapt and react depending on their contextual situation, considering their intrinsic 
nature but also elements of the environment (policy arenas, again), then ABM may be adequate. 
Finally, computational simulations are fit to include space in detail (Hammond 2015). Doing the 
same in equation-based models would make them quickly intractable. Although using deter- 
ministic computational code, simulations need not impose monotonicity and convexity (nor 
equilibrium) in order to be computed. 

Note that the definition of ABM and Epstein’s proposed motto (2007) include a necessary 
condition to understand a phenomena: being able to grow it — that is, replicate the mechanisms 
needed for the creation of patterns — although it may not be a sufficient one. The simulated 
model produced is but a candidate at explanation. There might be models that are unknown 
to the modeler that would serve the purpose of growing recognizable patterns. Automated 
processes, operated via machine learning, for instance, could produce other models that are 
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efficient in producing predictions. ABM, however, focuses on agents’ cognition and space-time 
descriptions that encompass reasonable parameters. “To explain a social pattern, one must show 
how the pattern could emerge on time scales of interest to humans in a population of cognitively 
plausible agents” (Joshua M. Epstein 2006, 1587). 

The definition of complex adaptive systems (Holland 1992; Furtado and Sakowski 2014) 
intrinsically implies that there is not just a single possible future trajectory for a system. Con- 
versely, a number of pathway alternatives may occur, depending on both the initial conditions 
of the system and the interactions among the system’s constituent parts. Although unpredictable, 
reasonable near-future states probabilities are possible, constrained by the assumptions made and 
the current status quo knowledge of the relevant mechanisms. In practice, scenarios — which 
alter main inputs and mechanisms of a modeled system in a controlled manner — contribute 
to aiding decision-making processes that take place independent of certain comprehension of 
future effects (Maier et al. 2016). 

Given this general conceptualization of agent-based modeling, we proceed to list some 
advantages and disadvantages of applying ABM to policy. Most of the advantages are bound to 
the definition of ABM. Agents, environment, interactions, space and time set a framework for 
analysis that systemically incorporate theory, data and tacit knowledge to provide computational 
experiments, counterfactuals and scenarios. The disadvantages depend on the specification of 
each one of those items, on the construction and implementation of the model and on its actual 
use for decision-making. 

Some of the advantages come from the contrast of typical equation-based methods. ABM 
proposes cognitive plausible agents (Joshua M. Epstein 2007) with limited rationality (Arthur 
1994), instead of working on aggregate measures (Edmonds and Meyer 2017) and agents that 
detain complete knowledge of the present and future in infinite lifetime style (Fagiolo and 
Roventini 2017). ABM uses an empirical spatially explicit environment (Taillandier et al. 2019), 
not one into which space is difficult to incorporate, except in a highly stylized way (Thisse et al. 
2021). Note, however, that instead of opposites, ABM and equation-based methods may be 
complementary (Grabner et al. 2017; Beaussier et al. 2019). 

In addition to those structural advantages, ABM has been shown to be useful to compare 
methodologies from distant disciplines (Chattoe-Brown 2021). The need for a formal (compu- 
tational) layout serves the purpose of laying the groundwork, which accepts additional modular 
levels and functions as a repository. Thus, ABM may help organize the multitude of inputs from 
different actors, interests, disciplines, methods and data. “A rise in interdisciplinary teams work- 
ing together to address pressing social challenges, leveraging the explosive growth of available 
data and computational power” (Buyalskaya et al. 2021, 1). Actually, explicitly involving stake- 
holders and making them hands-on modelers “increases the likelihood that the model will be 
used and will be fit for purpose” (Gilbert et al. 2018, 1). 

Another relevant advantage of ABM is its relative low-cost and fast in-silico implementation. 
Policy decision-making is necessary even in a crisis situation in which information and under- 
standing are lacking. Quickly prototyping might help develop a general sense of the effects of 
radical interventions. 

Disadvantages also abound when using ABM. A structural and intrinsic difficulty is to find 
the “appropriate level of abstraction” (Gilbert et al. 2018, 11). Complex models may include 
more details than necessary to describe a phenomenon and thus obscure relationships so that 
stakeholders and policymakers cannot pinpoint which inputs affect the outputs and by what 
order of magnitude. Conversely, the model may be so simple that the main mechanism associ- 
ated with the problem at hand is not present, thus rendering the model useless or even hazardous 
(Aodha and Edmonds 2017). 


274 


Simulation Modeling as a Policy Tool 


Kurtz and Snowden (2003) suggest that ABM is less equipped to handle people and organi- 
zations when compared with ABM applied to the analysis of natural systems. Specifically, they 
mention that humans may not be contained within a single identity or role; rather, humans are 
themselves complex systems. Kurtz and Snowden also criticize the fact that people would follow 
predetermined rules as they have free will and might follow collective opinions. Finally, Kurtz 
and Snowden suggest that people are able to act simultaneously on local and global scales and in 
the entire spectrum in between. 

Aodha and Edmonds (2017) present some specific recommendations when using ABM for 
policy analysis. Models in general, and ABM in particular, are loaded with scientists’ points of 
view, background and disciplinary formation, and those influence modeling assumptions and 
constitute a limiting of theoretical spectacles. Modelers and model users should also understand 
the model limitations and make sure that the model has been tested and checked thoroughly 
(Edmonds and Gershenson 2015). When the model lands on policymakers’ desks, Aodha and 
Edmonds (2017) suggest that it is likely that no third party has audited nor tested the model. 
It is common — given the complexity of the model and the tacit knowledge of its details and 
intricacies — that only a handful of people can properly run the model. Occasionally, not even 
the modeler but only the computer analyst can fully comprehend the model’s computational 
implementation (Galan et al. 2009). Further misuse of the model for policy, according to Aodha 
and Edmonds (2017), includes the aforementioned confusion between a model’s purpose, its 
goal and the context in which it was designed to be effective. 

A final disadvantage we present is the lack of a standardization (Beaussier et al. 2019). The 
generality and flexibility of the method contribute to a relatively lack of benchmark cases 
with example models varying in detail, implementation, documentation and description. 
Since the 2010s, some of this last disadvantage has become less relevant. In economics, for 
example, a number of ABM practices have risen providing minimum standards for macro- 
economic models (Dawid and Gatti 2018). In the social sciences, a strong community has 
emerged consolidating the debate on epistemological and ontological aspects of the meth- 
odology (Edmonds and Meyer 2017). In terms of communicating the models, a standard 
protocol was proposed in 2006 (Grimm et al. 2006), and then it was adopted by the com- 
munity, extended and adapted in more recent years (Grimm et al. 2010, 2020). Hammond 
(2015) lists some best practices. 

The conceptualization of ABM is not complete without the discussion of verification, cali- 
bration and sensitivity analysis and validation. There is not absolute consensus in the literature, 
especially about validation (Galan et al. 2017). Thus, we present a brief overview of each term. 

Verification relates to the formal codification of the model and to errors of computation. 
This happens when the programmer believes the computer is calculating a precise division, for 
example, but given internal mechanisms of the language, what actually happens is different. That 
may occur also when the program has run from beginning to end, apparently without throwing 
an error, only because the error is not one of syntax but of interpretation: an update of a state 
that should have happened but did not. Galan et al. (2009) also define errors as artefacts, which 
is the incorrect interpretation of the model. Artefacts take place when the modeler presumes a 
given assumption or process is generating a specific result, when other assumptions or mecha- 
nisms are the real influence. 

Calibration is the process of iteratively (and often in an automated manner) adjusting empiri- 
cal data to make the model compatible with observed phenomena. Some caution is warranted: 
data used for calibrating a model should not be the same data used for validating the model. 
When doing calibration, the modeler searches for the best value for parameters so that the model 
replicates empirical patterns and stylized facts. 
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Sensitivity analysis is somewhat similar as different configuration runs of the model are com- 
pared among themselves. However, the emphasis here is that, given a calibrated, verified model, 
what are the effects of a change in specific parameters? At times, the analysis illuminates explana- 
tory mechanisms and conditional scenarios (Gilbert et al. 2018). As a result, sensitivity analysis 
may work as a test of robustness or as an exploratory tool. 

Validation — although central to the modeling process — is a concept whose definition is 
more ample (Ngo and See 2012; Moss 2008; van Vliet et al. 2016). Moss (2008) and Galan 
et al. (2017) advocate that validation be attached to the purpose of the model. Hence, a model 
is valid when it fulfills the goal that was originally set. Validation is also defined as the “accurate” 
replication of a time-series set of data (Guerini and Moneta 2017). A partial compromise is 
that different modeling purposes demand different validation processes (Edmonds et al. 2019). 
Models that aim at predicting data would need to have proved themselves to have replicated 
out-of-sample empirical data that has not entered in the model, either as input or as calibration. 
A model proposed as a participatory analogy among stakeholders, policymakers and scientists 
might still be valid and function as a systemic template for knowledgeable interactions and com- 
munication without necessarily replicating existing data. 

Independently of which validation definition a modeler uses, the more aspects of the model 
that are validated the better. Thus, replicating and validating agent behaviors’ via a survey while 
simultaneously producing aggregate validation of empirical indicators (Haldane and Turrell 
2019) and a higher number of them provides more validation to the model. 


Applications of ABM for Policy 


Discussion 


ABM was propagated as a “promising feature” for scientific and policy analysis in the 2000s and 
2010s (Jovanovic et al. 2012): a possible candidate to surpass current best practices (Farmer et al. 
2015) and a likely tool to integrate economic and ecological models (Lamperti et al. 2019). 
Mainly, the promise was to assess “the relative merits of alternative policy prescriptions in meet- 
ing the policy objectives” (Gilbert et al. 2018, 2, emphasis added). 

The promising methodology has evolved into a varied array of examples. Lee et al. (2015) list 
works in ecology, economics, health care, sociology, geography, anthropology, archaeology, bio- 
terrorism, business, education, medical research, military tactics, neuroscience, political science, 
urban development and land use and zoology. More discipline-specific reviews may be found 
in the area of geography and cities (Heppenstall et al. 2012; Batty 2018), policy (Furtado et al. 
2015; Edmonds and Meyer 2017) and economics (Dawid and Gatti 2018). Here we present just 
a few examples that actually succeeded in the policy-applied criteria. 

Our first example that tries to improve “efficacy of policy response” (Carley et al. 2006, 282) 
comes from a model that incorporated actions and interactions of agents within their social 
and professional networks. The departing point of the model is the fact that diffusion and out- 
break of diseases happen amidst people as they go about, incorporating aspects at the physical, 
biological, social and economic environment. Carley et al. develop BioWar to assess numeri- 
cally the impact of policy interventions. BioWar functions as an integrated platform in which 
modular models are inserted in specific time points in the simulation, generating independent 
parts that function sequentially. A larger scope model — the “simulator state machine” — starts 
the process and at a given moment calls an “agent state machine”, which, in turn, eventually 
runs a “disease state machine”. In fact, the agent model receives inputs from data but also from 
five other sub-models: (1) disease, (2) geography, (3) weather, (4) attack and (5) communication 
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and technology. The model is validated against data on absenteeism and medical information. 
The validation runs in an automated tool in order to handle the complexity of the model. The 
automation allows for both validation and gain of mechanism understanding made via specifi- 
cally formulated queries. 

Kerr et al. (2021) implement this same idea of using ABM to evaluate policy interventions 
in a pandemic state in a number of countries across the world. COVASIM implements COVID 
progression, from susceptible to recovery or death for each individual agent. Agents interact in 
contact networks at home, in their communities, and at their school or workplace. Genera- 
tion of synthetic agents, households, schools and workplaces comes from population data. The 
authors state that they verified the model thoroughly, presenting a number of embedded tests. 
Calibration was used to approximate parameters of the model to real-world data, whereas vali- 
dation was applied upon actual use on a policy decision. COVASIM (Kerr et al. 2021) provides 
an array of policy interventions, including (1) physical distancing, masks and hygiene; (2) testing 
and diagnosis; (3) contact tracing; (4) isolation of positives and quarantine; and (5) vaccines and 
treatments. Treatment tests include expected reduction of the probability of progression of the 
disease. Specifically, COVASIM tested mobility restrictions imposed after the run of the model. 
“Despite a rapid increase of cases in the preceding weeks, the model predicted counterintuitively 
that even these modest mobility restrictions would be sufficient to stop the rise in cases, a pro- 
jection that turned out to be accurate” (Kerr et al. 2021, 24). At the same time, a counterfactual 
policy intervention, that of not having imposed additional restrictions — for obvious reasons — 
could not be tested but was predicted to have generated a three-times-higher infection rate. 

Gilbert et al. (2018) provide some examples of models that have been successful in changing 
policy parameters and being deployed in other occasions afterwards, such as Silent Spread and 
Exodis-FMD, INFSO-SKIN, the abstractor behaviour model. Silent Spread was first applied in 
2003 following the foot-and-mouth disease outbreak of 2001. 


The modelling was critical to the Government’s decision to relax the 20-day move- 
ment control to 6 days, subject to commitments from the livestock industry. The itera- 
tive, participatory development process generated an unprecedented level of ‘buy-in’ to 
the results in an area which had previously been marked by deep controversy. 

(Gilbert et al. 2018, 9) 


INFSO-SKIN model, in turn, was key to helping design a funding-guideline policy before it 
was actually implemented (Ahrweiler et al. 2015). The model compares a no-change-in-policy 
baseline with four alternatives. The modelers tested changes in scope, instrument, amount of 
funding and incentives to smaller groups’ participation. In short, the model simulates a market 
in which agents of varying size and complexity try to sell innovation to other users, trying to 
adapt, learn, cooperate and change its performance in the process to that they can increment 
their knowledge base. Together, agents associate in research consortia to make proposals and seek 
funding from well-delineated call criteria. The baseline scenario follows empirical data obtained 
for the Framework Programme 7 from the European Commission, whereas the previous six 
frameworks were used for calibration of the model. Ahrweiler et al. (2015) tested ten specific 
experiments with varying degrees of recommendations. According to Gilbert et al. (2018), 
INFSO-SKIN results informed the next funding cycle. Paier et al. (2017) also apply ABM to 
knowledge creation and policy recommendations. 

ABM in economics is far from consensus or mainstream (Fagiolo and Roventini 2017). 
However, the field has provided a benchmark of good practices (Dawid and Gatti 2018) and 
illustrations of models central banks use to inform decisions. Mostly, the models emphasize 
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macroprudential policies to reduce real estate market volatility. In a very specific context and 
with a heavily empirically calibrated model, stakeholders investigate the sensitivity of loan-to- 
value parameters to bound price volatility. A model developed and extended for Washington, 
DC, was applied later in the United Kingdom and evaluated in Denmark (Geanakoplos et al. 
2012; Baptista et al. 2016; Goldstein 2017; Carstensen 2015). 

Apart from these specific real estate market applications, Dawid and Gatti (2018) list appli- 
cations on fiscal and monetary policy, financial regulation and crisis management, labor mar- 
ket and regional growth. The authors emphasize that applications have become more specific 
and detailed since 2013, comparatively to more generic recommendations that were the norm 
before. Moreover, validation and testing have become more rigorous with a wider array of 
methodological tools available. 

The Bank of England (Baptista et al. 2016) and the Organization for Economic Coopera- 
tion and Development (OECD 2020) seem to have been supporting the use of integrated novel 
methodology such as ABM to tackle systemic challenges in the economy. A working paper pro- 
duced by the New Approaches to Economic Challenges (NAEC) (Naumann-Woleske 2021) — 
an initiative within the OECD — produced a review of ABM models that have helped make 
policy recommendations including (1) innovation and industrial policy, (2) macroeconomic 
policy, (3) financial regulation, (4) wealth and income inequality, (5) labor market policy and 
automation and (6) sustainability and decarbonization. 

The European Commission (EC) has also funded the Complexity Research Initiative for 
Systemic Instabilities (CRISIS) (Klimek et al. 2015). A recent breakthrough in validation and 
prediction mentioned by the OECD is the full simulation of the Austrian economy, made by 
Poledna et al. (2020). 

Poledna, Miess and Hommes aim at a full-scale, data-rich simulation, including all sectors 
and accounts of the economy. In a flexible model, without imposing equilibrium and with 
agents who make decisions based on partial information and limited cognition, the authors 
make an explicit effort to compare it with typical vector autoregression (VAR) models and 
dynamic stochastic general equilibrium (DSGE) models. The authors claim that the model is 
able to compete with VAR and DSGE models with the bonus of providing a forecast of disag- 
gregated sectoral variables and components of GDP. This responds to a need posed by Farmer 
and Foley (2009). 


Final Considerations 


This chapter describes the use of the agent-based model as a scenario tool to aid policymakers 
anticipate effects of policy intervention. The text frames policy as a complex system and presents 
the concept, advantages and limitations of ABM as a tool. We present a few applications of ABM 
for policy that were useful. Our take is that ABM has come from a potential and useful method- 
ological tool in the 2000s to a real, effective one in the 2010s, albeit less so for policy analyses. “The 
methodological difficulty is to bridge the gap between policy practice, often expressed in qualita- 
tive and narrative terms, and the scientific realm of formal models” (Ahrweiler et al. 2015, 1). 

The listings of actual use of ABM to policy intervention, although reasonable, are not over- 
whelming. However, ABM’s contribution varies from enabling the understanding of triggers 
and circumstances of social unrest (Jovanovic et al. 2012) to helping the development of theo- 
retical policy approaches (Klein 2021) or specifically indicating which epidemic intervention 
should be implemented and when (Kerr et al. 2021). But one must keep in mind the purposes 
of the model (Edmonds et al. 2019), its limitations (Aodha and Edmonds 2017) and the quality 
of its validation, given its context (Galan et al. 2009). 
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